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DETAILED ACTION 

1 . This action is responsive to the following communication: Amendment filed on 
3/6/07. This action is made final. 

2. Applicant amended claims 1, 16, 28 and 29. 



Claim Rejections - 35 USC §112 

3. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

4. Claim 36 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the written description requirement. The claim(s) contains subject matter, which 
was not described in the specification in such a way as to reasonably convey to one 
skilled in the relevant art that the inventor(s), at the time the application was filed, had 
possession of the claimed invention. Applicant does not reasonably convey how the 
paintbrush is scaled according to how the source image is scaled. If it is well known in 
the art, the applicant should provide evidence. Therefore, the rejection is maintained. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



Application/Control Number: 10/633,776 Page 3 

Art Unit: 2179 

6. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f)or(g) 
prior art under 35 U.S.C. 103(a). 

7. Claim 1 , 4, 7-13, 16, 18-23, 28, 29, and 30-37 are rejected under 35 U.S.C. 
103(a) as being unpatentable over "Photoshop 3," by Dayton et al (hereinafter Dayton), 
in view of Hsu et al (hereinafter Hsu) US Patent 6078701 . 

• In regards to claims 1,16, and 29, Dayton teaches a computer- 
implemented process, a system, computer-readable medium, and a graphical 
user interface for creating a composite image, comprising using a computer to 
perform the following process actions: applying one or more filters to the image 
to create one or more new intermediate images (pg. 54); selecting one of the 
original images in the image stack or an intermediate image as a source image 
(pg. 54); and selecting pixels from the source image to be added to a composite 
image to create a final composite image (pg. 54). However, Dayton does not 
specifically teach and image stack or inputting an image stack comprising a stack 
of original images, wherein the pixel position of each original image in the image 
stack is defined in a three dimensional coordinate system, and wherein two 
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dimensions describe the dimensions of each image in the image stack, and the 
third dimension describes the time an image was captured. 
Hsu teaches a method and apparatus for performing local to global multiframe 
alignment to construct mosaic images. Hsu further teaches an image stack and 
inputting an image stack comprising a stack of original images taken from the 
same point of view, wherein the pixel position of each original image in the image 
stack is defined in a three dimensional coordinate system, and wherein two 
dimensions describe the dimensions of each image in the image stack, and the 
third dimension describes the time an image was captured (column 18 lines 30- 
46 and column 16 line 44-column 17 line 9). It would have been obvious to one 
of ordinary skill in the art at the time of the invention to modify the method and 
apparatus of Dayton to include the time of image capture in order to create three 
dimensions out of a two-dimensional image stack. One would have been 
motivated to make such a combination in order to create a three dimensional 
image out of a two-dimensional image stack, based on time of image capture. 
• In regards to claims 4 and 1 9, Dayton teaches wherein said process 
action of applying a filter comprises applying a median filter that returns the 
median pixel luminance along a span of the image, wherein a span is a set of 
image pixels at the same location in all images of the image (pg. 140). 
Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
one of ordinary skill in the art at the time of the invention to modify the method 
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and apparatus of Dayton to include an image stack in order to create three 
dimensions out of a two-dimensional image. One would have been motivated to 
make such a combination in order to create a three dimensional image out of a 
two-dimensional image stack, for accurately aligned images (abstract). 

• In regards to claims 7 and 20, Dayton teaches, wherein said process 
action of applying a filter comprising applying a maximum luminance filter that 
returns the pixel with the maximum luminance along a span of the image, 
wherein a span is the set of image pixels at the same location in all images of the 
image (pg. 140). 

Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
one of ordinary skill in the art at the time of the invention for the same reasons 
stated above (see claims 4 and 19 supra). 

• In regards to claims 8 and 21 , Dayton teaches wherein said process 
action of applying a filter comprises applying a maximum contrast filter that 
returns the pixel that has the highest contrast in a small neighborhood around it 
along a span of the image, wherein a span is the set of image pixels at the same 
location in all images of the image (pg. 76). 

Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
one of ordinary skill in the art at the time of the invention for the same reasons 
stated above (see claims 4 and 19 supra). 
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• In regards to claim 9, Dayton teaches wherein said small neighborhood is 
5 by 5 pixels (pg. 83). 

• In regards to claims 10, Dayton teaches wherein said-process action of 
applying a filter comprises applying a temporal smoothing filter that returns a 
weighted blend of a current image and the images before and after it, for a given 
span of the image, wherein a span is the set of image pixels at the same location 
in all images of the image (pg 139). 

Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
one of ordinary skill in the art at the time of the invention for the same reasons 
stated above (see claims 4 and 1 9 supra). 

• In regards to claim 1 1 , Dayton teaches wherein said process action of 
applying a filter comprises applying a temporal sharpening filter that returns a 
pixel in the current image modified by the difference of the pixels in the images 
before and after the current image for a given span of the image, wherein a span 
is the set of image pixels at the same location in all images of the image (pg 
136). 

Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
one of ordinary skill in the art at the time of the invention for the same reasons 
stated above (see claims 4 and 19 supra). 
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• In regards to claim 12, Dayton teaches wherein said process action of 
applying a filter comprises applying a high dynamic range filter that combines 
different exposures over a span of the image, wherein a span is the set of image 
pixels at the same location in all images of the image (pg. 76). 

Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
one of ordinary skill in the art at the time of the invention for the same reasons 
stated above (see claims 4 and 19 supra). 

• In regards to claim 13, Dayton teaches wherein the high dynamic range 
filter extracts exposure information associated with the original images that 
comprise the image (pg. 76). 

Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
one of ordinary skill in the art at the time of the invention for the same reasons 
stated above (see claims 4 and 19 supra). 

• In regards to claim 18, Dayton teaches wherein said module to apply a 
filter applies a high dynamic range luminance filter that comprises sub-modules 
to: compute a radiance value for each pixel in said image; map the radiance 
value for each pixel to its luminance value by mapping red, green and blue 
channels to a display to match the luminance (pg. 76). 

Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
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one of ordinary skill in the art at the time of the invention for the same reasons 
stated above (see claims 4 and 19 supra). 

• In regards to claim 22, Dayton teaches wherein said module to apply a 
filter applies a high dynamic range filter that comprises sub-modules to: compute 
a radiance value for each pixel in said image; map the radiance values for each 
pixel back to a set of display values via a tone-map (pg. 76). 

Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
one of ordinary skill in the art at the time of the invention for the same reasons 
stated above (see claims 4 and 19 supra). 

• In regards to claim 23, Dayton teaches wherein said tone-map is user- 
defined (pg. 76). 

• In regards to claim 28, Dayton teaches a computer-readable medium 
having computer executable instructions stored thereon for editing an image, 
said computer executable instructions operable to: apply a filter to the image to 
create an intermediate images (pg. 54); select one of the images in the image or 
an intermediate image to serve as a source image for creating a new composite 
image (pg. 54); and select pixels from the source image to create a final 
composite image (pg. 54). However, Dayton does not specifically teach an image 
stack or instructions operable to input an image stack comprising a stack of 
original images, taken from the same point of view, wherein the pixel position of 
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each in the image stack is defined in a three dimensional coordinate system, 
wherein one dimension is time. 

Hsu teaches a method and apparatus for performing local to global multiframe 
alignment to construct mosaic images. Hsu further teaches an image stack and 
instructions operable to input an image stack comprising a stack of original 
images, taken from the same point of view, wherein the pixel position of each in 
the image stack is defined in a three dimensional coordinate system, wherein 
one dimension is time (column 18 lines 30-46 and column 16 line 44-column 17 
line 9). It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the method and apparatus of Dayton to include the time 
of image capture in order to create three dimensions. One would have been 
motivated to make such a combination in order to create a three dimensional 
image out of a two-dimensional image stack, based on time of image capture. 
• In regards to claim 30, Dayton teaches wherein said user creates said 
intermediate image by applying at least one filter to the image and users said 
intermediate image as a source image (pg. 54). 

Hsu teaches the above limitatations (see claims 1,16, and 29 supra). Hsu further 
teaches an image stack (column 18 lines 30-35). It would have been obvious to 
one of ordinary skill in the art at the time of the invention for the same reasons 
stated above (see claims 4 and 19 supra). 
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• In regards to claim 31 , Dayton teaches wherein parts of said source image 
are transferred to said composite image by transferring pixels from the source 
image to the composite image (pg. 54). 

• In regards to claim 32, Dayton teaches wherein said transfer of pixels from 
said source image to said composite image is based on a one-to-one 
correspondence regardless of whether the user initiates pixel transfer from the 
source image or the composite image (pg. 83). 

• In regards to claims 33 and 35, Dayton teaches further comprising a 
paintbrush function that transfers some pixels from said source image to said 
composite image (pg. 176). 

• In regards to claim 34, Dayton teaches wherein a radius of pixel transfer is 
user-defined (pg. 83 and 140). 

• In regards to claim 36, Dayton teaches wherein scaling the source image 
or the composite image scales paint brush function (pg. 76). 

• In regards to claim 37, Dayton teaches wherein a highest resolution image 
available is used when transferring pixels using the paintbrush function even 
when the source image or composite image is scaled (pg. 35). 

8. Claims 2, 5, and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Dayton and Hsu as applied to claims 1, 4, 7-13, 16, 18-23, 28, 29, and 30-37 
above, and further in view of Wise et al (hereinafter Wise), US Patent 6130676 
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• In regards to claim 2, Dayton and Hsu teach the above limitations in the 
claims above (see claims 1,4, 7-13, 16, 18-23, 28, 29, and 30-37 supra). 
However, Dayton and Hsu do not specifically teach wherein the process action of 
inputting an image stack comprises inputting an image stack wherein said 
original images are defined in a Cartesian coordinate system. 

Wise teaches an image composition system and process using layers. Wise 
further teaches wherein the process action of inputting an image stack comprises 
inputting an image stack wherein said original images are defined in a Cartesian 
coordinate system (Figure 8). It would have been obvious to one of ordinary skill 
in the art at the time of the invention to modify the method and apparatus of 
Dayton and Hsu in order to include images defined in a Cartesian coordinate 
system to be easily understood. One would have been motivated to make such a 
combination in order to allow all users to easily understand how the images are 
stacked according to a familiar coordinate system. 

• In regards to claim 5, Dayton and Hsu teach the above limitations in the 
claims above (see claims 1,4, 7-13, 16, 18-23, 28, 29, and 30-37 supra). 
However, Dayton and Hsu do not specifically teach wherein said process action 
of applying a filter comprises applying a maximum histogram filter that returns the 
pixel with the minimum sum of squared distances in red, green, blue color space 
to all other pixels along a span of the image stack, wherein a span is the set of 
image pixels at the same location in all images of the image stack. 
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Wise teaches an image composition system and process using layers. Wise 
further teaches wherein said process action of applying a filter comprises 
applying a maximum histogram filter that returns the pixel with the minimum sum 
of squared distances in red, green, blue color space to all other pixels along a 
span of the image stack, wherein a span is the set of image pixels at the same 
location in all images of the image stack (column 5 lines 1-14). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to 
modify the method and apparatus of Dayton and Hsu to include a histogram filter 
that returns the pixel with the minimum sum of squared distances in red, green, 
blue color space in order to maximize color levels. One would have been 
motivated to make such a combination in order to make a customized color 
palette. 

• In regards to claim 5, Dayton and Hsu teach the above limitations in the 
claims above (see claims 1, 4, 7-13, 16, 18-23, 28, 29, and 30-37 supra). 
However, Dayton and Hsu do not specifically teach wherein said process action 
of applying a filter comprises applying a minimum histogram filter that returns the 
pixel with the minimum sum of squared distances in red, green, blue color space 
to all other pixels along a span of the image stack, wherein a span is the set of 
image pixels at the same location in all images of the image stack. 
Wise teaches an image composition system and process using layers. Wise 
further teaches wherein said process action of applying a filter comprises 
applying a minimum histogram filter that returns the pixel with the minimum sum 
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of squared distances in red, green, blue color space to all other pixels along a 
span of the image stack, wherein a span is the set of image pixels at the same 
location in all images of the image stack (column 5 lines 1-14). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to 
modify the method and apparatus of Dayton and Hsu to include a histogram filter 
that returns the pixel with the minimum sum of squared distances in red, green, 
blue color space in order to minimize color levels. One would have been 
motivated to make such a combination in order to make a customized color 
palette. 

9. Claims 3 and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dayton and Hsu as applied to claims 1, 4, 7-13, 16, 18-23, 28, 29, and 30-37 above, 
and further in view of Jodoin et al (hereinafter Jodoin), US Patent 5493419. 

• In regards to claim 3, Dayton and Hsu teach the above limitations in the claims 
above (see claims 1, 4, 7-13, 16, 18-23, 28, 29, and 30-37 supra). However, 
Dayton and Hsu do not specifically teach wherein said process action of applying 
a filter comprises applying a slice filter wherein said filter returns an image in said 
image stack. 

Jodoin teaches stack filters for 1-to-N bit image processing in electronic printers. 
Jodoin further teaches wherein said process action of applying a filter comprises 
applying a slice filter wherein said filter returns an image in said image stack 
(Abstract). It would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the method and apparatus of Dayton and Hsu to 
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include a slice filter. One would have been motivated to make such a 
combination in order to use the filter to better format the image. 

• In regards to claim 17, Dayton and Hsu teach the above limitations in the claims 
above (see claims 1,4, 7-13, 16, 18-23, 28, 29, and 30-37 supra). However, 
Dayton and Hsu do not specifically teach wherein said module to apply a filter 
applies a slice (x,y) filter wherein for each (x, y) said filter returns a pixel at depth 
z from said image stack.. 

Jodoin teaches stack filters for 1-to-N bit image processing in electronic printers. 
Jodoin further teaches wherein said module to apply a filter applies a slice (x,y) 
filter wherein for each (x, y) said filter returns a pixel at depth z from said image 
stack (Abstract). It would have been obvious for the same reasons stated above 
(see claim 3 supra). 

10. Claims 14, 26,and 27 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Dayton and Hsu as applied to claims 1 , 4, 7-13, 16, 18-23, 28, 29, and 30-37 
above, and further in view of Okamoto et al (hereinafter Okamoto), US Patent 5754618. 

• In regards to claims 14 and 26, Dayton and Hsu teach the above limitations in 
the claims above (see claims 1, 4, 7-13, 16, 18-23, 28, 29, and 30-37 supra). 
However, Dayton and Hsu do not specifically teach wherein said process action 
of applying a filter comprises applying a surface filter that operates on a given 
surface through the image stack. 

Okamoto teaches an image processing apparatus and method for favorably 
enhancing continuous boundaries, which are affected by noise. Okamoto further 
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teaches said process action of applying a filter comprises applying a surface filter 
that operates on a given surface through the image stack (column 19 lines 12- 
21). It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the method and apparatus of Dayton and Hsu to include a 
surface filter in order to filter via surface of the image. One would have been 
motivated to make such a combination in order to use the filter to better format 
the image. 

• In regards to claim 27, Dayton and Hsu teach the above limitations in the claims 
above (see claims 1,4, 7-13, 16, 18-23, 28, 29, and 30-37 supra). However, 
Dayton and Hsu do not specifically teach wherein said surface embedded in the 
image stack is user-defined. 

Okamoto teaches an image processing apparatus and method for favorably 
enhancing continuous boundaries, which are affected by noise. Okamoto further 
teaches wherein said surface embedded in the image stack is user-defined 
(Abstract). It would have been obvious for the reasons stated in the above claim 
(see claims 14 and 26 supra). 
1 1 . Claims 15, 24 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Dayton and Hsu as applied to claims 1 , 4, 7-13, 16, 18-23, 28, 29, and 30-37 
above, and further in view of Chuang et al (hereinafter Chuang), "Video Matting of 
Complex Scenes". 

• In regards to claims 15, 24, and 25, Dayton and Hsu teach the above limitations 
in the claims above (see claims 1,4, 7-13, 16, 18-23, 28, 29, and 30-37 supra). 
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However, Dayton and Hsu do not specifically teach wherein said process action 
of applying a filter comprises applying a mat filter that produces a mat of a given 
portion of the image stack, wherein the mat is an image of transparency values 
that will modify the source image when it is used for creating said composite 
image. 

Chuang teaches video matting of complex scenes. Chuang further teaches 
wherein said process action of applying a filter comprises applying a mat filter 
that produces a mat of a given portion of the image stack, wherein the mat is an 
image of transparency values that will modify the source image when it is used 
for creating said composite image (page 1 , introduction). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to modify the 
method and apparatus of Dayton and Hsu to include a mat filter to manipulate 
the background and foreground. One would have been motivated to make such 
a combination in order to use the filter to better format the image. 
12. Claim 38 is rejected under 35 U.S.C. 103(a) as being unpatentable over Dayton 
and Hsu as applied to claims 1, 4, 7-13, 16, 18-23, 28, 29, and 30-37 above, and further 
in view of Funayama et al (hereinafter Funayama), US Patent 63891 55. 

• In regards to claim 38, Dayton and Hsu teach the above limitations in the claims 
above (see claims 1, 4, 7-13, 16, 18-23, 28, 29, and 30-37 supra). However, 
Dayton and Hsu do not specifically teach further comprising a paint brush 
function that transfers all pixels associated with a face from said source image to 
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said composite image when said paint brush function is used to select a portion 
of said face. 

Funayama teaches an image processing apparatus. Funayama further teaches 
further comprising a paint brush function that transfers all pixels associated with 
a face from said source image to said composite image when said paint brush 
function is used to select a portion of said face (Figure 4). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to modify the 
method and apparatus of Dayton and Hsu to include transferring all pixels 
associated with a face from said source image to said composite image when 
said paint brush function is used to select a portion of said face in order to select 
a portion of the face. One would have been motivated to make such a 
combination in order to better format the image. 

Response to Arguments 
Applicant argues that Dayton does not teach an intermediate image. Applicant 
quotes page 3 paragraph 4 of the specification, "Filters may be applied to tile 3D image 
stack, or a portion thereof , to create one or more new 2D intermediate images. A filter is 
a function that operates on the 3D image stack to create a 2D image. An intermediate 
image is one created by running a filter on the image stack." Applicant argues that in 
contrast, Dayton teaches a technique for applying filters to a single image. 

Examiner disagrees. As the applicant has clearly pointed out, an intermediate 
image by applying filters "to a 3D image stack, or a portion thereof." One image can be 
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considered a portion of an image stack and once filtered would be an intermediate 
image. Therefore, Dayton teaches the above limitations. 

Applicant argues that Dayton does not teach creating high dynamic range 
images. Examiner disagrees. Dayton clearly creates a high dynamic range image 
clearly shown on page 76. Applicant defines a high dynamic range to be, "the 
HighDynamicRange (x, y, map(R — *Y)) filter assumes the input stack has images with 
different exposures and exposure information. This filter computes a radiance value for 
each pixel and then tone maps the radiance back to the gamut of the monitor (e.g., 0 to 
255) via a user defined tone map (automatic tone mapping can also be used). The map 
can be adjusted in real time as the user observes the resulting intermediate image 
(specification page 4)." Dayton shows filtering the radiance value (e.g. 0-255, RGB 
levels, see margin on page 76). Therefore, Dayton meets the above claimed limitations. 

Applicant argues that Dayton does not teach combining images captured under 
different lighting conditions, removing objects from images, and combining images 
captured at multiple points in time or with different focal lengths, and creating a variety 
of special effects using the image stack. Examiner disagrees. These limitations are not 
in the claim language and therefore claim interpretations are left up to the broadest 
interpretation of the examiner. 

Applicant's arguments with respect to claims 1-38 dealing with reference, Xu, 
have been considered but are moot in view of the new ground(s) of rejection. 

Conclusion 
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Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Shashi K. Becker whose telephone number is 571 -272- 
8919. The examiner can normally be reached on Mon-Fri 8:30-5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Weilun Lo can be reached on 571-272-4847. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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